Sparse high-dimensional linear regression. Estimating squared error and a phase transition

We consider a sparse high-dimensional regression model where the goal is to recover k-sparse unknown binary vector β∗ from n noisy linear observations of form Y=Xβ∗+W∈Rn X∈Rn×p has i.i.d. N(0,1) entries and W∈Rn N(0,σ2) entries. In high signal-to-noise ratio regime sublinear sparsity regime, while order sample size needed information-theoretically known be n∗:=2klogp/log(k/σ2+1), no polynomial-time algorithm succeed unless n>nalg:=(2k+σ2)logp. this work, we offer series results investigating multiple computational statistical aspects recovery task in n∈[n∗,nalg]. First, establish novel information-theoretic property MLE problem happening around n=n∗ samples, which coin as an “all-or-nothing behavior”: when n>n∗ it recovers almost perfectly support β∗, if n0 nCnalg disappears. geometric “disconnectivity” property, initially appeared theory spin glasses suggest algorithmic occurs. Finally, using certain technical obtained transition, additionally various positive negative interest, including failure LASSO with access success simple local search method samples.